{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Image Data Explanation Benchmarking: Image Multiclass Classification" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This notebook demonstrates how to use the benchmark utility to benchmark the performance of an explainer for image data. In this demo, we showcase explanation performance for partition explainer on an Image Multiclass Classification model. The metrics used to evaluate are \"keep positive\" and \"keep negative\". The masker used is Image Masker with Inpaint Telea.\n", "\n", "The new benchmark utility uses the new API with MaskedModel as wrapper around user-imported model and evaluates masked values of inputs." ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "import json\n", "\n", "import numpy as np\n", "from tensorflow.keras.applications.resnet50 import ResNet50, preprocess_input\n", "\n", "import shap\n", "import shap.benchmark as benchmark" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Load Data and Model" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "model = ResNet50(weights=\"imagenet\")\n", "X, y = shap.datasets.imagenet50()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Class Label Mapping" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "url = \"https://s3.amazonaws.com/deep-learning-models/image-models/imagenet_class_index.json\"\n", "with open(shap.datasets.cache(url)) as file:\n", " class_names = [v[1] for v in json.load(file).values()]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Define Score Function" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [], "source": [ "def f(x):\n", " tmp = x.copy()\n", " if len(tmp.shape) == 2:\n", " tmp = tmp.reshape(tmp.shape[0], *X[0].shape)\n", " preprocess_input(tmp)\n", " return model(tmp)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Define Image Masker" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [], "source": [ "masker = shap.maskers.Image(\"inpaint_telea\", X[0].shape)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Create Explainer Object" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "explainers.Partition is still in an alpha state, so use with caution...\n" ] } ], "source": [ "explainer = shap.Explainer(f, masker, output_names=class_names)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Run SHAP Explanation" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "", "version_major": 2, "version_minor": 0 }, "text/plain": [ "HBox(children=(FloatProgress(value=0.0, max=248.0), HTML(value='')))" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stderr", "output_type": "stream", "text": [ "\r", "Partition explainer: 50%|█████████████████ | 1/2 [00:00" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "shap.image_plot(shap_values)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Get Output Class Indices" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [], "source": [ "output = f(X[1:3]).numpy()\n", "num_of_outputs = 4\n", "sorted_indexes = np.argsort(-output, axis=1)\n", "sliced_indexes = np.array([index_list[:num_of_outputs] for index_list in sorted_indexes])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Define Metrics (Sort Order & Perturbation Method)" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [], "source": [ "sort_order = \"positive\"\n", "perturbation = \"keep\"" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Benchmark Explainer" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "", "version_major": 2, "version_minor": 0 }, "text/plain": [ "HBox(children=(FloatProgress(value=0.0, max=1.0), HTML(value='')))" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEGCAYAAABo25JHAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/d3fzzAAAACXBIWXMAAAsTAAALEwEAmpwYAAAu70lEQVR4nO3deXyU9bX48c+Z7AlLQsKeQIKAgCwBIkhLrbtYrWhBhW5atba2Wmvrba1t7a21v9brvVq1tRWXamuLrbYqtrhUEVFBSICwCwbISoSQFcg6mfP7YyZxCFkmyUwmkznv1ysvZ57nO8+cJ8E5891FVTHGGBO+HMEOwBhjTHBZIjDGmDBnicAYY8KcJQJjjAlzlgiMMSbMRQY7gO5KSUnR9PT0YIdhjDEhZfPmzUdVdXh750IuEaSnp5OTkxPsMIwxJqSISEFH56xpyBhjwpwlAmOMCXOWCIwxJsyFXB+BMSY0NDU1UVxcTH19fbBDCSuxsbGkpqYSFRXl82ssERhjAqK4uJjBgweTnp6OiAQ7nLCgqpSXl1NcXExGRobPr7OmIWNMQNTX15OcnGxJoA+JCMnJyd2uhVkiMMYEjCWBvteT37klAhM0qso/NhdTUlUX7FCMCWuWCExQqCo/f2U3339+G3/e0OE8F2N67aWXXkJE+PDDD1uPrV27lssuu+ykctdddx0vvPAC4O7ovvPOO5k0aRJz5sxhwYIFvPrqq6dc++DBg8yfP5+JEydyzTXX0NjYeEqZTZs2kZmZSWZmJrNmzeLFF19sPVdVVcXSpUuZMmUKU6dOZcOGDQDk5uZy1llnkZmZSVZWFps2bTrpmtnZ2URGRrbG21uWCEyfa0kCT6/PRwSKKmqDHZIZwFauXMnChQtZuXKlz6/56U9/SmlpKTt37mTLli289NJLHDt27JRyP/zhD7n99tvJy8sjKSmJJ5988pQy06dPJycnh9zcXF577TW+8Y1v4HQ6AbjttttYtGgRH374Idu2bWPq1KkA/OAHP+BnP/sZubm53HPPPfzgBz9ovV5zczM//OEPueiii7r7q+iQJQLT51qSwI0LM1g4MYXiSksEJjCOHz/Oe++9x5NPPslzzz3n02tqa2t5/PHHeeSRR4iJiQFg5MiRXH311SeVU1XWrFnD0qVLAbj22mt56aWXTrlefHw8kZHuAZr19fWtbfjV1dWsW7eOG264AYDo6GgSExMBdzt/TU1Na7kxY8a0Xu+RRx5hyZIljBgxwsffQtds+KjpU1sLK3l6fT7XfSqdH186lbte3Mkbhz4OdlgmwH7+yi52H6rx6zWnjRnCzz5/RqdlXn75ZRYtWsTkyZNJTk5m8+bNzJ07t9PX5OXlMW7cOIYMGdJpufLychITE1s/5FNTUykpKWm37MaNG7n++uspKCjgz3/+M5GRkRw8eJDhw4fzta99jW3btjF37lweeughEhIS+M1vfsPFF1/MHXfcgcvlYv369QCUlJTw4osv8vbbb5Odnd1pfN1hNQLTp/78QQEJ0RHccfHpiAipSXGUn2jkRIMz2KGZAWjlypUsW7YMgGXLlrU2D3U0siZQo5zmz5/Prl27yM7O5le/+hX19fU4nU62bNnCzTffzNatW0lISODXv/41AL///e958MEHKSoq4sEHH2ytNXz3u9/lvvvuw+Hw70e31QhMn6k80ci/tpdydVYqg2I836KS4gAoqapj8sjBwQzPBFBX39wDoaKigjVr1rBjxw5EhObmZkSE+++/n+TkZCorK08pn5KSwsSJEyksLKSmpqbTWkFycjJVVVU4nU4iIyMpLi5m7NixncY0depUBg0axM6dO0lNTSU1NZX58+cDsHTp0tZE8Mwzz/DQQw8BcNVVV3HjjTcCkJOT05rYjh49yurVq4mMjOSKK67o0e+ohdUITJ95YXMxjU4XXz5rfOuxtGHxgHUYG/974YUX+MpXvkJBQQH5+fkUFRWRkZHBu+++y6RJkzh06BB79uwBoKCggG3btpGZmUl8fDw33HADt912W+sooLKyMp5//vmTri8inHvuua0jd5555hkWL158ShwHDx5s7RwuKCjgww8/JD09nVGjRpGWlsbevXsBeOutt5g2bRoAY8aM4Z133gFgzZo1TJo0qfVa+fn55Ofns3TpUh599NFeJwEIcCIQkUUisldE8kTkznbOXyciZSKS6/m5MZDxmOBxuZS/bCwga3wSU0Z98i2rpUZQXGlzCYx/rVy5kiuvvPKkY0uWLGHlypXExMTw7LPP8rWvfY3MzEyWLl3KE088wdChQwG49957GT58ONOmTWP69Olcdtll7dYO7rvvPh544AEmTpxIeXl5axPOqlWruPvuuwF47733mDVrFpmZmVx55ZU8+uijpKSkAO6O3y996UvMnDmT3Nxc7rrrLgAef/xxvv/97zNr1izuuusuVqxYEbDfE4CoamAuLBIB7AMuBIqBbGC5qu72KnMdkKWqt/h63aysLLWNaULPun1lfPWpTTy0LJPFmZ9Un1WVKT99ja8uGM+PL50WxAiNv+3Zs6d1OKTpW+397kVks6pmtVc+kDWCeUCeqh5Q1UbgOeDUepMJC89+UEByQjSLpo866XhLh3FRhdUIjAmWQCaCsUCR1/Niz7G2lojIdhF5QUTSAhiPCZKSqjre3HOYq7LSiImMOOV82rB4iqusj8CYYAl2Z/ErQLqqzgT+AzzTXiERuUlEckQkp6ysrE8DNL339PsHERG+fNa4ds9bjWDgClTTs+lYT37ngUwEJYD3N/xUz7FWqlquqg2ep08A7c70UNUVqpqlqlnDhw8PSLAmMGrqm1i5qYhLZ4wmNSm+3TKpSfFU1zVRU9/Ux9GZQIqNjaW8vNySQR9q2Y8gNja2W68L5DyCbGCSiGTgTgDLgC96FxCR0apa6nl6ObAngPGYIFi5sZDjDU5uOntCh2XSPAmiuKKOaWN831XJ9G+pqakUFxdjtfi+1bJDWXcELBGoqlNEbgFeByKAp1R1l4jcA+So6irgOyJyOeAEKoDrAhWP6XuNThd/fD+fT52WzPSxQzss98kQ0lqmjel8Wr8JHVFRUd3aJcsET0BnFqvqamB1m2N3ez3+EfCjQMZggudf2w/xcU09v1oyo9NyLZPKbC6BMcER7M5iM0CpKivWHWDyyEGcM7nzfp2k+CjioyMoslVIjQkKSwTG78qONfDtv27hw4+P8fXPTOhyIa+WuQRWIzAmOGzROeM3qso/t5Rwz792U9fYzH9dfDpL5vjWaZWWFG/rDRkTJJYIjN+8tvNjvv/8NuaOT+K+JTOZOGKQz69NTYpj08EKVNU2PDemj1kiMH7zx/fzGTcsnr9/YwERju59mKcNi+dYg5PquiYS46MDFKExpj3WR2D8Yk9pDZvyK/jKWeO7nQTAViE1JpgsERi/+NOGAmKjHFyV1b2JLC1aZh2H2/7FLpdSWl1ns29NUFnTkOm16tomXtpawuJZY3vcrNMyuzjc1hy651+7eXp9PqOGxHLWhGEsOC2ZM9OHkZGS0GFfSbNLcUjgtlU04ccSgem15zcXUdfUzFcWjO+6cAeGxEUyJDaSA0dP+DGy/u2VbYd4en0+l0wfRYRDeC/vKC/lHgIgZVA0c8cnccaYoUwZNZgxie7O9DUfHmHjwXJcCvHREQyKiWTE4BjGJsWRmhTP3PFJnD1pOHHRp67yakxHLBGYXnG5lGc/KGDu+KROl5Hoiogwe1wSOfkVfoyu/9pfdpw7/7GdueOTeHj5bKIiHKgq+8uOk51fSXZ+BVsKKnlj92G8W40mjRjEtQvSiY2K4HiDk+MNTg7X1PNh6THe3HOEFesOEBcVwdmTU0hLisfpUpwuF41OFw1OF/VNzThEiIuOICE6krjoCGKjIoiNcuAQobbByYnGZuqammlyumhsduFsdl/D5YljUEwkg2IiiY+JIDrCQYRDcIjQ7Hkvp0txuRSXgksVhwgRDvfP8EHupDU20f2TGB9lNZt+wBKB6ZV3846SX17L7RdO7vW15mUM4/7X91J+vIHkQTF+iK5/qmts5lvPbiEmKoLfftGdBMCdDCeOGMzEEYNZPs+9ZPeJBif7Dh+jsKKWzLRExicndHjdpmYXGw9U8Pquj3lzz2HW7TtKZIQQFeEgOsJBbJSDmMgIXKrUej7saxud1De5Wq8hAvFREcRFuz/koyIdRDqESIcDh0NQVU40Ojle7+REQ/NJCQJo/cCP8Hz4C+5k0KzqSSgn94XERjkYPTSOuKgIFFu2uiu3njeJS2eO9vt1LRGYXlm5sZDkhGgumd77f5zzM4YBkJ1fecpOZgNFUUUt3/1bLvuOHOOZr81j9NC4TssnxEQye1wSs8cldXntqAgHCyelsHBSCr+4YrrPMakqDU4XLlXioiK6/Q3d/e1f3R/8nbxWVamsbaKkso6SqlpKquopraqjtLqeBmczIIiA1Q86lhATmCY/SwSmx8qONfDmnsNcvzCD6MjeD0CbmZpITKSDTQcrQj4R1NQ38a9tpZQda2Bm6lBmpSXyft5R7npxByg8snw2Z3exBlNfERFio3r+AeNwCA4fPr5FhGEJ0QxLiGZGas+bEY3/WSIwPfbC5mKcLuXqLP/sMBod6WDOuCQ2Hiw/6XgozTbeU1rDinUHeHVn6UlNLi1mj0vk4WWzW1dcNaY/sERgekRV+Vt2IfPSh3VrKYmuzMsYxsNrPqKmvokhse5Nar757GaaXcqjX5rrl5pHoBwoO841j21AFZbMSeXqrDROGzGInSXVbCuqIjYqgi/OH9faJ2BMf2GJwPTIBwcqyC+v5TvnT/LrdednDEMVNudXcu6UEWTnV/D6rsMA/PSlnfx6yYx+WTuormvixmdyiIxw8PK3P33SN/6zJiRz1oTkIEZnTOfsq4npkeeyCxkSG8nnZvh3BMPscUlERQgbD7qHkT76dh7DEqK5cWEGf8sp4ol3D/r1/fzB2ezi1pVbKaqs5fdfmmPNPibkWI3AdFtVbSOv7vyY5Wem9aqTsT1x0RHMTE1k08Fydh2q5u29ZfzXxadz82dP41B1Hf/v1T1kpCRwwbSRfn3f3rj/jb2s21fGr78wg/n2zd+EIKsRmG5pcDbzs1W7aHS6uObMcQF5j3kZw9heXM0Db+xjcEwkXz5rPA6H8H9XZXLGmCHc+c8d1Dc1B+S9u6vsWAN/fC+fpXNTWTYvML8PYwLNEoE5haqy+1AND7/1Eff+azfbiqpQVcqONfDFxzfycu4hvnfh5IBtND8vYxhOl/LWh0f4yoLxDI1zdxrHRUfwo0umcvR4A//cUhKQ9+6ulZsKaWx2cfM5pwU7FGN6zJqGzElW7yjll//eQ0lVHSIQ5XDwxHsHmThiELUNTipqG/ndF+cEZHZji6zxSTjEPUHq+oUZJ5371GnJzBg7lMffPcA1Z6b1aMlrf2l0unj2gwI+O3k4pw3338gpY/qaJQLTytns4r9X7WJIXBT3LZnBeVNGEh3pYPWOUv6xuZhIh7Diq1m9WlPIF4Njo7gicywThieQ0mapCRHhG5+dwC1/3cobuz7mEj93VnfHqztLOXKsgfuWpgctBmP8wRKBafXOvjKOHGvgF1dM5+IzPpnZu3zeuNa1b/rKA9dkdnjukumjGZ+8lz+8s59F00d1azjp8QYng2L888/+6fX5ZKQk8NlJ/WOGsDE9ZX0EptVz2UWkDIrhvCkjgh1KpyIcwtc/M4FtxdVsOFDe9Qs8/rP7MLPveYM9pTW9jiG3qIqthVVcu8DdkW1MKLNEYAA4UlPPmg+PsHRuakjMfF06N5WUQdHc9lwui36zjrP/522u++OmTlev/MvGApqatddzEVSVx989wKCYSJb6aXkNY4Kp//8fb/rEC1uKaXYp15wZGh9ssVER/PSyaUwZNZhxw+JJGxbH2r1lZOdXtlv+yLF61u0rY3BMJK9sO8SRY/WnlPno8DHufnkn8375Ju9+VNbudRqdLu78xw7+vb2UrywY77dmJmOCyf4VG8+6QUXMz3BvkRgqFmeOZXHmWABqG52cee+bvLC5iHme5ay9vbz1EC6Fh5ZncsMzOTy7oYDvXXQ64J4LcPvfcnkv7yjREQ4cDng+p5jPtGn7Lz/ewM3PbmFTfgW3njeR2y/o/R4MxvQHViMwfHCggoLyWpbNC43aQHvioyO5dOZo/r29lNpG50nnVJV/bClmVloi500ZyflTRvLsxkLqm5qpa2zmxmeyySmo4AeLTmfDj87j8zPHsHbvEZqaP1k9tNmlXP3YBrYVV/HQsky+f9Hp1jdgBgxLBIa/5xQxODbSL5vLBNPSuWmcaGxm9Y6PTzq+u7SGDz8+xtI57trDDQszqDjRyD+3lPCd57ayvaSaR5bP4VvnTCR5UAwXTBtJTb2TbK9tM9d9VMb+shP8z9KZrbUQYwaKgCYCEVkkIntFJE9E7uyk3BIRURHJCmQ85lTOZhdv7TnMxWeM8vu6QX3tzPQk0pPjeT6n6KTj/9hcQlSE8PlZYwA4a8Iwpo4ews9W7eQ/uw/zs8umcaHX2kWfmZRCdKSDN3cfaT32fE4Rw/y0E5sx/U3AEoGIRAC/Ay4BpgHLRWRaO+UGA7cBGwMVi+lYblEVNfVOzj29fw8Z9YWIsHRuKhsPVlBYXgu49/F9ObeE86eMJDE+urXc1z+TQVOzcv2nM7ju0yfPXo6PjuTTpyXz5p7D7u0VTzTy5u4jLM4c06/3QzCmpwL5r3oekKeqB1S1EXgOWNxOuV8A9wGnDuMwAbd2bxkRDmHhpJRgh+IXX5iTigg8v7mIzQWV3PXPHZSfaGTJ3NSTyl05eyyv3LKQn1w6td3rXDBtJIUVteQdOc7LuSU0Nru4am7o9qEY05lAJoKxgHcdvdhzrJWIzAHSVPXfnV1IRG4SkRwRySkra39Yn+mZd/aVMTstsXVht1A3JjGOhRNTeGRNHkt+v56Xtx1iyZxUzjn95BFAIsKM1KEddvieP8XdVPSfPYd5fnMx08cOCdgie8YEW9CGj4qIA3gAuK6rsqq6AlgBkJWV1fGMIdMtZcca2FFSzR0XDaxhkLedP4mhcVGcN2UEF04byeDY7ie5UUNjmTF2KM+sz+dwTQM/v/yMAERqTP8QyERQAnjXpVM9x1oMBqYDaz1rxYwCVonI5aqaE8C4jMe6fe7a1TkDoH/AW1b6MLLST51L0F0XTB3Jg2/uIzrCweLMMX6IzJj+KZBNQ9nAJBHJEJFoYBmwquWkqlaraoqqpqtqOvABYEmgD63dV0bKoBimjbYmj/ZcMM2dIC8845OOZmMGooDVCFTVKSK3AK8DEcBTqrpLRO4BclR1VedXMIHU7FLe/aiM86eMtIlRHZg2egj/dfHpLJo+quvCxoSwgPYRqOpqYHWbY3d3UPacQMZiTratuIqq2qZTOlHNJ0SEb587MdhhGBNwNig6TK3dW4ZD3JOnjDHhzRJBmFq3r4zMtERr+zbGWCIIR/VNzew6VM38CcnBDsUY0w9YIghDuw7V0NSsZKYlBjsUY0w/YIkgDG0tdG/eMtsSgTEGSwRhKbeoirGJcYwYEhvsUIwx/YAlgjCUW1RlzULGmFaWCMJM2bEGiivrLBEYY1pZIggzuUVVAMwelxjUOIwx/YclgjCztbCSSIcwfezQYIdijOknLBGEmdyiKqaMHhzy21IaY/zHEkEYaXYp24urmZ2WFOxQjDH9iCWCMJJ35DjHG5zWUWyMOUmXiUBErvLlmOn/covcE8kyraPYGOPFlxrBj3w8Zvq5rYVVDI2LIiM5IdihGGP6kQ73IxCRS4DPAWNF5GGvU0MAZ6ADM/6XW1TFrLRE24jGGHOSzjamOQTkAJcDm72OHwNuD2RQxv9UlYNHT7Bwou0/YIw5WYeJQFW3AdtE5C+qajWAEFdxopEGp4sxiXHBDsUY08/4slXlRyKibQ+q6oQAxGMCpLS6HoAxibbQnDHmZL4kgiyvx7HAVcCwwIRjAuVQVR0Ao4dajcAYc7IuRw2parnXT4mq/ga4NPChGX9qqRGMthqBMaaNLmsEIjLH66kDdw3Bl5qE6UcOVdcRFSGkJMQEOxRjTD/jywf6/3k9dgL5wNUBicYETGlVPaOGxtrQUWPMKbpMBKp6bl8EYgKrtLrO+geMMe3yZYmJZBF5WES2iMhmEXlIRJL7IjjjP4eq6hkz1PoHjDGn8mWJieeAMmAJsNTz+G+BDMr4V7NLOVxTz2ibQ2CMaYcvfQSjVfUXXs/vFZFrAhWQ8b+jxxtwutRqBMaYdvlSI3hDRJaJiMPzczXweqADM/5jcwiMMZ3xJRF8Hfgr0OD5eQ74hogcE5Gazl4oIotEZK+I5InIne2c/6aI7BCRXBF5T0Sm9eQmTOdsDoExpjO+jBoa3JMLi0gE8DvgQqAYyBaRVaq626vYX1X1D57ylwMPAIt68n6mYy01gjFWIzDGtMOXUUNv+XKsHfOAPFU9oKqNuGsSi70LqKp3jSIBOGVNI9N7pdX1xEY5SIyPCnYoxph+qLP9CGKBeCBFRJKAlplIQ4CxPlx7LFDk9bwYmN/O+3wb+B4QDZznW9imO0qr6xgzNA4Rm0xmjDlVZzWCb+Deh2AKsMXzeDPwMvBbfwWgqr9T1dOAHwI/aa+MiNwkIjkiklNWVuavtw4bh6rqrX/AGNOhDhOBqj6kqhnAHaqa4fUzS1V9SQQlQJrX81TPsY48B1zRQSwrVDVLVbOGDx/uw1sbbzar2BjTGV/mEVSLyFfbHlTVP3Xxumxgkohk4E4Ay4AvehcQkUmq+pHn6aXARxi/amp2ceRYg21IY4zpkC+J4Eyvx7HA+bibijpNBKrqFJFbcM85iACeUtVdInIPkKOqq4BbROQCoAmoBK7twT2YThyuqUcVm0xmjOmQL8NHb/V+LiKJuJtxuqSqq4HVbY7d7fX4Np+iND32yRwCqxEYY9rny4Sytk4AGf4OxATGJ3MIrEZgjGmfLxvTvMIn4/sdwDTg74EMyviP1QiMMV3xpY/gf70eO4ECVS0OUDzGz0qr6hgcG8mgGNtUzhjTPl+ahgqBwZ6fUksCoeVQdb0tLWGM6VRnM4uHAE8Ac4FtnsOZIrIZuKHN8hCmnyqtrrPJZMaYTnVWI3gY2A1MUtUvqOoXgNOAHfhxZrEJrNKqeptMZozpVGcNx59W1eu8D6iqAveIiE38CgG1jU7KTzSSmmSJwBjTsZ4MH4VPFqAz/VhhRS0A44bFBzkSY0x/1lkiWC8id0ubJStF5KfAhsCGZfyhoNydCMYnWyIwxnSss6ahW4EngTwRyfUcywS2AjcENizjD4XlViMwxnStw0TgGRV0lYichnsSGcBuVd3fJ5GZXiusqGVIbCSJ8dHBDsUY04/5stbQfsA+/ENQQUUt45MTgh2GMaaf62lnsQkBheUnrFnIGNMlSwQDVLNLKa6sY5x1FBtjutDZzOJhnb1QVSv8H47xl0NVdThdynirERhjutBZH8Fm3KuOtjdnQIEJAYnI+EXrHAKrERhjutDZqCHbcyCEFdjQUWOMj7rsIxC3L3smkiEi40RkXuBDM71RWFFLVITYOkPGmC750ln8KLCATzaePwb8LmARGb8orDhBWlI8EQ5bDcQY0zlfdiuZr6pzRGQrgKpWiojNUOrnCsprrX/AGOMTX2oETSISgWe7ShEZDrgCGpXpFVWlsLzW+geMMT7xJRE8DLwIjBCRXwLvAf8voFGZXqmqbeJYg9MSgTHGJ74sMfEXz65k5+MeSnqFqu4JeGSmxwoqWlYdteUljDFd83VC2RFgpfc5m1DWfxWUnwBs+WljjG98nVA2Dqj0PE7EvaG9zTPop1qWn05LskRgjOlah30EqpqhqhOAN4HPq2qKqiYDlwFv9FWApvsKK2oZMTiGuOiIYIdijAkBvnQWn6Wqq1ueqOqrwKcCF5LpLffy01YbMMb4xpdEcEhEfiIi6Z6fHwOHAh2Y6Tn30FHrKDbG+MaXRLAcGI57COmLwAjPMdMP1TY6+bim3oaOGmN85svw0QrgNhEZ7H6qx329uIgsAh4CIoAnVPXXbc5/D7gRcAJlwPWqWtCN+E0b2fmVAGSOSwxuIMaYkOHLonMzPMtL7AR2ichmEZnuw+sicK9JdAnuPY+Xi8i0NsW2AlmqOhN4Afif7t6AOdn6vKNERQhnpicFOxRjTIjwpWnoMeB7qjpeVccD3wdW+PC6eUCeqh5Q1UbgOWCxdwFVfVtVaz1PPwBSfQ/dtGf9/nJmpyURH+3LMlLGGONbIkhQ1bdbnqjqWsCXnsixQJHX82LPsY7cALza3gkRuUlEckQkp6yszIe3Dk9VtY3sPFTNpyYmBzsUY0wI8SURHBCRn3qNGvoJcMCfQYjIl4Es4P72zqvqClXNUtWs4cOH+/OtB5QPDlSgCp86LSXYoRhjQogvieB63KOG/un5Ge451pUSIM3rearn2ElE5ALgx8Dlqtrgw3VNB9bvP0pcVASZaYnBDsUYE0J8GTVUCXynB9fOBiaJSAbuBLCMTza3AUBEZuPug1ikqkd68B7Gy/r95ZyZMYzoSF/yuzHGuHW26Nyqzl6oqpd3cd4pIrcAr+MePvqUqu4SkXuAHFVdhbspaBDwvIgAFHZ1XdO+wzX15B05zlVzrb/dGNM9ndUIFuDu7F0JbMS94Fy3eJamWN3m2N1ejy/o7jVN+9bvPwpY/4Axpvs6SwSjgAtxzyL+IvBvYKWq7uqLwEz3rM8rZ2hcFNPGDAl2KMaYENPZ6qPNqvqaql4LnAXkAWs9zT2mH1FV1u8vZ8GEZNus3hjTbZ12FotIDHAp7lpBOp9sW2n6CVXlN29+RElVHTefc1qwwzHGhKDOOov/BEzH3cb/c1Xd2WdRGZ/UNTZzxwvb+Pf2UpbMSeXqrLSuX2SMMW10ViP4MnACuA34jmdUD7g7jVVVrTE6iOqbmlm2YgPbS6r50SVTuOnsCXj9jYwxxmcdJgJVtcHo/diG/eVsK67m/qUzucpqAsaYXrAP+xC1o6QaEbhkxuhgh2KMCXGWCELUjpJqMlISGBRjq4waY3rHEkGI2lFczcyxQ4MdhjFmALBEEILKjjXwcU090y0RGGP8wBJBCNpZUg3ADEsExhg/sEQQgrYXuzuKz7BEYIzxA0sEIWhHSTUTrKPYGOMnlghC0M6SamsWMsb4jSWCEHPkWD0f19QzIzUx2KEYYwYISwQhxjqKjTH+ZokgxOwornF3FNu+A8YYP7FEEGJ2lFRx2vBBJFhHsTHGTywRhJgd1lFsjPEzSwQh5EhNPYdrGmxGsTHGrywRhJCtRVUAzEy1RGCM8R9LBCFkfd5R4qIimGVDR40xfmSJIIS8l3eU+ROGER1pfzZjjP/YJ0qIKK2uY3/ZCRZOTAl2KMaYAcYSQYh4P68cgE9bIjDG+JklghDxft5RkhOiOX3k4GCHYowZYCwRhABV5b28o3xqYgoOhwQ7HGPMAGOJIAR8dOQ4ZccaWDgxOdihGGMGoIAmAhFZJCJ7RSRPRO5s5/zZIrJFRJwisjSQsYSSZpeSd+R46/P3PjoKWP+AMSYwApYIRCQC+B1wCTANWC4i09oUKwSuA/4aqDhC0dPr87nggXf41at7aHYp7+cdJT05ntSk+GCHZowZgAK5ctk8IE9VDwCIyHPAYmB3SwFVzfeccwUwjpDz2s5SYiIdPPbOAfYfOcHGgxUszhwT7LCMMQNUIBPBWKDI63kxML8nFxKRm4CbAMaNG9f7yPqxyhONbC6o5JZzJ5IyOIafv7KbZpfa/AFjTMCExFrGqroCWAGQlZWlQQ4noN7eewSXwvlTRzIrLZGMlARe2FzM2ZOHBzs0Y8wAFchEUAKkeT1P9RwznXhrzxGGD45pXWr6M5OG85lJlgSMMYETyFFD2cAkEckQkWhgGbAqgO8X8hqdLt7ZV8YFU0fYfAFjTJ8JWCJQVSdwC/A6sAf4u6ruEpF7RORyABE5U0SKgauAx0RkV6DiCQUbD5ZzvMHJ+VNGBjsUY0wYCWgfgaquBla3OXa31+Ns3E1GBnezUEykw+YLGGP6lM0s7idUlTf3HGbhxBTioiOCHY4xJoxYIugn9h0+TnFlHedPtWYhY0zfskTQT6z58AgA508dEeRIjDHhxhJBP7F+/1FOHzmYkUNigx2KMSbMWCLoBxqdLnLyK1lwmq0uaozpe5YI+oFtxVXUNTVz1gRLBMaYvmeJoB/4YH85IjA/Y1iwQzHGhCFLBP3AhgPlTBk1hKSE6GCHYowJQ5YIgqzB2czmgkoWWLOQMSZILBEE2dbCKhqcLusoNsYEjSWCINuwvxyHwDzrHzDGBIklgiDbcKCcM8YMZWhcVLBDMcaEKUsEQVTf1ExuYZU1CxljgsoSQRBtKaiksdllHcXGmKCyRBBE7+8/SoRDyEpPCnYoxpgwZokgSFSVf20vZX7GMAbHWv+AMSZ4LBEEybbiagrKa7kic2ywQzHGhDlLBEHy0tYSoiMdLJoxKtihGGPCnCWCIHA2u/jX9lLOO30EQ6xZyBgTZJYIgmD9/nKOHm/gitljgh2KMcZYIgiGl3MPMTg2knNOt93IjDHBZ4mgj9U3NfP6ro+5ZPooYqNsk3pjTPBZIuhjb+05wvEGJ4tttJAxpp+wRNCHVJWn1x9k5JAY243MGNNvWCLoQ69sLyU7v5LvXjCZCIcEOxxjjAEsEfSZ2kYnv1q9hzPGDOHqrLRgh2OMMa0igx1AuPjD2v2UVtfzyPLZVhswxvQrViPoA0UVtTy27gCLM8eQlW4b0Bhj+hdLBAFWWl3HrSu34hDhzkumBDscY4w5RUATgYgsEpG9IpInIne2cz5GRP7mOb9RRNIDGU9fe2PXx1zy0LvsO3yM/71qFqOHxgU7JGOMOUXA+ghEJAL4HXAhUAxki8gqVd3tVewGoFJVJ4rIMuA+4JpAxdRbqkptYzNVdU0cqqqjsLyW4so6BsVGMmF4AhnJCVTUNpJbWMUHB8p5Y/dhzhgzhEeWz2bC8EHBDt8YY9oVyM7ieUCeqh4AEJHngMWAdyJYDPy35/ELwG9FRFRV/R3Mnz8o4LdrPiLS4SA60nFSh62qogAKCjQ6XTQ1u39aywAnGpw0NfsW2pihsXzzs6dx+4WTiIm0GcTGmP4rkIlgLFDk9bwYmN9RGVV1ikg1kAwc9S4kIjcBNwGMGzeuR8GMGxbPOZNH0ORy4WxWnC4XgtfoHQFxvxdREUJMpINIhwPxKpIQE8nQuCiGxkUxemgsacPiGZsYx4kGJweOnuBg2QmGxEUye1wSI4fE9ihOY4zpayExfFRVVwArALKysnpUW/js5OF8dvJwv8bVIjYqguRBMZxpI4KMMSEokJ3FJYD3zKlUz7F2y4hIJDAUKA9gTMYYY9oIZCLIBiaJSIaIRAPLgFVtyqwCrvU8XgqsCUT/gDHGmI4FrGnI0+Z/C/A6EAE8paq7ROQeIEdVVwFPAn8WkTygAneyMMYY04cC2kegqquB1W2O3e31uB64KpAxGGOM6ZzNLDbGmDBnicAYY8KcJQJjjAlzlgiMMSbMSaiN1hSRMqCghy9Poc2s5TBg9xwe7J7DQ2/uebyqtjurNuQSQW+ISI6qZgU7jr5k9xwe7J7DQ6Du2ZqGjDEmzFkiMMaYMBduiWBFsAMIArvn8GD3HB4Ccs9h1UdgjDHmVOFWIzDGGNOGJQJjjAlzAzIRiMgiEdkrInkicmc752NE5G+e8xtFJD0IYfqVD/f8PRHZLSLbReQtERkfjDj9qat79iq3RERUREJ+qKEv9ywiV3v+1rtE5K99HaO/+fBve5yIvC0iWz3/vj8XjDj9RUSeEpEjIrKzg/MiIg97fh/bRWROr99UVQfUD+4lr/cDE4BoYBswrU2ZbwF/8DxeBvwt2HH3wT2fC8R7Ht8cDvfsKTcYWAd8AGQFO+4++DtPArYCSZ7nI4Iddx/c8wrgZs/jaUB+sOPu5T2fDcwBdnZw/nPAq7h31z0L2Njb9xyINYJ5QJ6qHlDVRuA5YHGbMouBZzyPXwDOF/HenTjkdHnPqvq2qtZ6nn6Ae8e4UObL3xngF8B9QH1fBhcgvtzz14HfqWolgKoe6eMY/c2Xe1ZgiOfxUOBQH8bnd6q6Dvf+LB1ZDPxJ3T4AEkVkdG/ecyAmgrFAkdfzYs+xdsuoqhOoBpL7JLrA8OWevd2A+xtFKOvynj1V5jRV/XdfBhZAvvydJwOTReR9EflARBb1WXSB4cs9/zfwZREpxr3/ya19E1rQdPf/9y6FxOb1xn9E5MtAFvDZYMcSSCLiAB4ArgtyKH0tEnfz0Dm4a33rRGSGqlYFM6gAWw48rar/JyILcO96OF1VXcEOLFQMxBpBCZDm9TzVc6zdMiISibs6Wd4n0QWGL/eMiFwA/Bi4XFUb+ii2QOnqngcD04G1IpKPuy11VYh3GPvydy4GVqlqk6oeBPbhTgyhypd7vgH4O4CqbgBicS/ONlD59P97dwzERJANTBKRDBGJxt0ZvKpNmVXAtZ7HS4E16umFCVFd3rOIzAYew50EQr3dGLq4Z1WtVtUUVU1X1XTc/SKXq2pOcML1C1/+bb+EuzaAiKTgbio60Icx+psv91wInA8gIlNxJ4KyPo2yb60CvuoZPXQWUK2qpb254IBrGlJVp4jcAryOe8TBU6q6S0TuAXJUdRXwJO7qYx7uTpllwYu493y85/uBQcDznn7xQlW9PGhB95KP9zyg+HjPrwMXichuoBn4L1UN2dquj/f8feBxEbkdd8fxdaH8xU5EVuJO5imefo+fAVEAqvoH3P0gnwPygFrga71+zxD+fRljjPGDgdg0ZIwxphssERhjTJizRGCMMWHOEoExxoQ5SwTGGBPmLBGYfklEmkUkV0R2isjzIhIfhBjOEZFPdXDuv0XkjjbH8j1j9/uUiKzt6UQ5EXlaRJb6OyYTWiwRmP6qTlUzVXU60Ah805cXeWaK+8s5QLuJwJiBxBKBCQXvAhNFJMGzVvsmz9rziwFE5DoRWSUia4C3RGSQiPxRRHZ41mtf4il3kYhsEJEtnlrGIM/xfBH5uef4DhGZIu49Kr4J3O6pmXzG12BFJF1E9ojI4549Ad4QkTjPubUi8qCI5HjKnCki/xSRj0TkXq9rvCQimz2vv8lzLMLzDX6nJ87b27yvw3P+Xk/Z+0Uk2/M7+IanjIjIb8W9vv+bwIie/1nMQDHgZhabgcXzDf8S4DXc6yStUdXrRSQR2OT5MAP3+u0zVbVCRO7DPe1+hucaSZ4mm58AF6jqCRH5IfA94B7P64+q6hwR+RZwh6reKCJ/AI6r6v/2IPRJwHJV/bqI/B1YAjzrOdeoqlkichvwMjAX9wz3/SLyoGcm8PWee4kDskXkH0A6MNZTS8LzO2gRCfwF9xr2v/Qkj2pVPVNEYoD3ReQNYDZwOu51+0cCu4GnenB/ZgCxRGD6qzgRyfU8fhf3siDrgcu92uZjgXGex/9R1ZY13C/Aa9kQVa0Ukctwf/i971liIxrY4PV+//T8dzPwBR/i62hKfsvxg6raEv9m3B/iLVqWv9gB7GpZJ0ZEDuBeTKwc+I6IXOkpl4Y7sewFJojII8C/gTe8rvkY8HdV/aXn+UXATK/2/6Gea5wNrFTVZuCQpxZlwpwlAtNf1alqpvcBcX+CL1HVvW2OzwdOdHE9wZ0slndwvmU11mZ8+/+iHGi7GchgoMrzX+/VXZuBuHbey9WmnAuIFJFzcCezBapaKyJrgVhPQpsFXIy72epq4HrPa9cD54rI/6lqPe77vVVVX/cOUEJ8G0cTGNZHYELJ68CtnoTQsqJqe/4DfLvliYgk4V599NMiMtFzLEFEJnfxfsdwf6i3Zx3u2slgz/W+AGzzfNPuraFApScJTMG9hHbLaqIOVf0H7mYu771qn8S9GNnfPc1prwM3i0iU57WTRSTBE/c1nj6E0bi3MDVhzhKBCSW/wL0K43YR2eV53p57gSRPp+o24FxVLcO9Sc1KEdmOu1loShfv9wpwZXudxaq6Hfgt8J6nCeubwI09u61TvIa7ZrAH+DXuJAbuXajWet7vWeBHbWJ6APd+xX8GnsDd/r9F3JugP4a7pvMi8JHn3J84uXnMhClbfdQYY8Kc1QiMMSbMWSIwxpgwZ4nAGGPCnCUCY4wJc5YIjDEmzFkiMMaYMGeJwBhjwtz/B5bNlV656QK3AAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "sequential_perturbation = benchmark.perturbation.SequentialPerturbation(\n", " explainer.model, explainer.masker, sort_order, perturbation\n", ")\n", "xs, ys, auc = sequential_perturbation.model_score(shap_values, X[1:2], indices=sliced_indexes[0])\n", "sequential_perturbation.plot(xs, ys, auc)" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.9" } }, "nbformat": 4, "nbformat_minor": 4 }